Human Machine Interface System Research Articles

Smart Printed Triboelectric Wearable Sensor with High Performance for Glove-Based Motion Detection.

In a world increasingly driven by data, wearable triboelectric nanogenerators (TENGs) offer a convenient way to monitor and collect information about human body motions. To meet the demands of the large-scale production of wearable TENGs, material selection to realize a high conversion efficiency and simplify the fabrication process remains a challenge. To address these issues, we present a simple-structured wearable printed arc-shaped triboelectric sensor (PATS) for finger motion detection by leveraging inkjet printing technology. In this regard, pressure sensors composed of diverse materials based on dielectric-dielectric and metal-dielectric structures in contact-separation mode were fabricated and compared. Thanks to the unique characteristics of the silver nanoparticle (Ag-NP)-printed layer and silicon rubber (SR), the SR-Ag PATS shows a high peak-to-peak voltage of 14.15 V and a short-circuit current of 0.78 μA. The proposed sensor with the capability of accurately identifying finger motions at various bending angles suggests promising application potential in glove-based human-machine interface (HMI) systems.

Platform Supporting Intelligent Human–Machine Interface (HMI) Applications for Smart Machine Tools

As the Internet of Things, artificial intelligence, and the fourth industrial revolution advance, smart factories and machines increasingly gain intelligent features that enable the integration of more sophisticated functionalities. Approaches to achieving this intelligence involve both internal systems, such as human–machine interface (HMI), and external systems, such as big data platforms and cloud services. Although current research leans toward studying external systems, accomplishing intelligent functions through such means poses more challenges in achieving real-time responses during machining processes than using internal systems. When intellectualizing machine tools through internal HMI systems, three critical issues must be addressed. First, HMI functions are structured to depend on the HMI itself, leading to a ripple effect where a problem occurring in one HMI function impacts the entire system. Second, owing to differences in development tools and programming languages, the interconnectivity between functions developed by multiple stakeholders to be loaded onto the HMI may suffer, leading to potential inefficiencies and increased maintenance costs. Third, although various types of computer numerical control (CNC) machines need to communicate with the HMI function, the diverse communication methods and development tools used by each CNC manufacturer result in identical intelligent functions being developed separately for each CNC type. To address these challenges, this study proposes an innovative HMI platform capable of executing and developing various intelligent functions. The HMI platform and its major components are designed and implemented through component-based development (CBD). Subsequently, the performance and effectiveness of the platform are validated using quality attribute scenarios.

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

Head and Voice-Controlled Human-Machine Interface System for Transhumeral Prosthesis

Head and Voice-Controlled Human-Machine Interface System for Transhumeral Prosthesis

Recent Progress in Flexible and Wearable All Organic Photoplethysmography Sensors for SpO2 Monitoring.

Flexible and wearable biosensors are the next-generation healthcare devices that can efficiently monitor human health conditions in day-to-day life. Moreover, the rapid growth and technological advancements in wearable optoelectronics have promoted the development of flexible organic photoplethysmography (PPG) biosensor systems that can be implanted directly onto the human body without any additional interface for efficient bio-signal monitoring. As an example, the pulse oximeter utilizes PPG signals to monitor the oxygen saturation (SpO2 ) in the blood volume using two distinct wavelengths with organic light emitting diode (OLED) as light source and an organic photodiode (OPD) as light sensor. Utilizing the flexible and soft properties of organic semiconductors, pulse oximeter can be both flexible and conformal when fabricated on thin polymeric substrates. It can also provide highly efficient human-machine interface systems that can allow for long-time biological integration and flawless measurement of signal data. In this work, a clear and systematic overview of the latest progress and updates in flexible and wearable all-organic pulse oximetry sensors for SpO2 monitoring, including design and geometry, processing techniques and materials, encapsulation and various factors affecting the device performance, and limitations are provided. Finally, some of the research challenges and future opportunities in the field are mentioned.

(Invited) ultrasensitive Low Cost Flexible Sensors and Bioinspired Sutureless Anastomosis Devices

I will present our efforts for investigating flexible electronic sensors for potential health monitoring applications and bioinspired sutureless anastomosis devices for easier, faster, and safer surgery.Highly sensitive flexible tactile sensors are of continuing interest for various applications including human–machine interface systems, internet of things, and wearable health monitoring devices. Current technologies for high sensitivity piezoresistive sensors tend to rely on costly materials and/or fabrication methods, limiting accessibility and scalability. Here, we report a facile sacrificial casting–etching method to synthesize nanoporous carbon nanotube/polymer composites for ultra-sensitive and low-cost piezoresistive pressure sensors. Our synthesis method overcomes the limitations of the traditional solution-dip-coating method for adhering nanoscale conductive materials to the nanoscale porous surface. Importantly, we show ultra-high sensitivity with a strain gauge factor over 300, which is ∼50 times higher than that of traditional CNT-based piezoresistive sensors. For practical tactile sensing applications, we demonstrate that the sensors can detect both gentle pressures (1 Pa–1 kPa) and low pressures (1 kPa–25 kPa) with a fraction of the cost. Our nanoporous polymer composite could contribute to expanding the scope of using nanocomposite sensors for medical applications from its easy tunability for sensing diverse range of tactile signals.Anastomosis, the surgical connection of adjacent tubular organs is a foundational surgical skill for many surgical procedures. The current standard of anastomosis is manually suturing two tubular structures together around an opening with fine sutures. However, it requires several years of surgical training, a long time (60 to 90 minutes per anastomosis) for procedure, and, at times, specialized equipment, facility and staffs. Thus, it is an expensive (up to $35,000 per procedure).Moreover, even in the hands of skilled surgeons, the anastomosis can be complicated by leakage or thrombosis. To address current challenges, we investigated a sutureless anastomosis device inspired by rose prickles. I will present our approach to design, fabricate, and test devices. Our device showed the failure force comparable to that of the handsewn suture (4.9 N) and no leakage up to 45 mL/min flow rate, well above the physiologic blood flow rate. Compared with handsewn anastomosis, the device resulted in minimum deformation of the anastomotic site. From in-vivo non-survival porcine studies, the device showed successful anastomosis (< 5 min per anastomosis) with no leaking for both arterial and venous anastomoses. We envision our findings can contribute to enabling easier, faster, and safer anastomosis, and making anastomotic techniques more accessible to a broad range of clinicians, researchers, and patients across the world.

Human Machine Interface System for Filling and Capping Process

In this research, bottle filling and capping processes are performed simultaneously in the pharmaceutical factory by using the PC based human machine interface (HMI) control system. In the traditional method of bottle filling process, the bottles are placed onto a conveyor and only one bottle is filled at a time. So, this method is time consuming and expensive as wide working place and many human labors is required. In this research, the proposed design can solve this problem without any complex control by using the HMI automation control system. The monitoring and running conditions in the packaging system are shown on the monitor of computer by using the composed of Visual Basic.Net and Mikro C. It also includes the volume of medical powder is limited by changing the size of filling funnel which is set up of shutter at the top and bottom of funnel. The entire system is more flexible and time saving. A prototype is implemented by using the DC motor, sensing devices; limit switches, peripheral interface controller (PIC) device and serial port communication. These control systems is very flexible, cost effective, space efficient and reduce complexity. PC based HMI control is used for monitoring the process.

Design of PLC control system for cascading tanks controlling

The article deals with the design of a PLC control system for controlling cascade tanks in the technological process of liquid mixing. The mixing process is carried out automatically according to information from individual sensors. The system also includes elements for manual control of the technological system. A human machine interface system was designed to operate the system. The simulation showed the correctness of the designed system.

Fabrication of multifunctional cotton textile with battery waste- derived graphene oxide for enhanced joule heating and electromagnetic interference shielding

Nowadays, the research on wearable electronics have received tremendous attraction because of their potential applications in personalized health monitoring and treatment, energy conversion and storage, and human-machine interface system. Herein, we report a facile route for the fabrication of electrically conductive cotton fabric with excellent joule heating and high electromagnetic shielding performances using graphene oxide (GO) and silver nitrate (AgNO3). The GO used in this study is exclusively synthesized from spent batteries in order to minimize the environmental pollution. The surface morphology, elemental analysis, electrical conductivity, thermo-heating behavior and electromagnetic shielding performance have been studied systematically. Due to the high electrical conductivity, the GO-Ag coated cotton with 5 wt% of GO reached high surface temperature of 117.8°C within 35 s, and also it exhibits high electromagnetic interference shielding efficiency value of 79.08 dB. The high flexibility, excellent conductivity, electromagnetic shielding efficiency and joule heating performance of GO-Ag coated cotton fabric suggesting that the GO synthesized from spent batteries will be a potential and valuable resource for the new generation of wearable electronics.

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

A speed limit advisory system provided by in-vehicle HMI considering auditory perception characteristics for connected environment

In-vehicle HMI (Human Machine Interface) systems are well studied due to their great application prospect, but the recent development of connected technologies raises more possibilities that this technology provides more accurate speed limit warnings to ensure driver safety according to specific vehicle and road parameters. To capitalize on the potential of in-vehicle HMI systems to provide auditory and visual warnings to drivers and to improve adoption and utilization of this warning information. To this end, we designed a speed limit advisory system (SLAS) that recommended reduced speeds for hazardous sections of road, communicated to drivers by their in-vehicle HMI systems. Frontal distance models of single and gradient auditory warning were implanted into the systems to determine the warning timing. We then tested these SLASs in a high-fidelity simulator in order to evaluate their utility. 70 participants (mean age = 28.7 years; 36 male) drove through a simulated 15 km stretch of one-way two-lane rural road. We evaluated five different warning modes: single visual (SV), single audio visual (SAV), gradient visual (GV), gradient audio visual (GAV) or control (roadside sign). Participants using the SLAS slowed down significantly compared to the control group before entering the curve, with drivers receiving GAV warnings driving lowest speeds. Less experienced drivers exhibited higher compliance with GAMs (GV or GAV). The SLAS was generally considered useful though not by drivers with>10 years of driving experience. Drivers tended to prefer singular warnings to gradated warnings, which were rated as equally preferable to roadside signage. Eye tracking data indicated that none of the SLAS modes were overly distracting to the drivers. Overall, we find that the SLAS improves driver safety while producing no obvious distraction. The main difficulty lies in persuading experienced drivers to use the system. In a word, the results of this study provide validation for the concept of HMI warning system using connected technology in hazardous road areas and help determine the appropriate timing of in-vehicle warnings.

Fiber-Based Triboelectric Nanogenerator for Mechanical Energy Harvesting and Its Application to a Human-Machine Interface.

Mechanical energy harvesters including piezoelectric nanogenerators, electromagnetic generators and triboelectric nanogenerators (TENG) used to convert the mechanical motion into electricity are more and more important in the recent decades. Specifically, the fiber-based TENG (FTENG) has gained considerable favors due to its flexibility, light weight, and high environmental tolerance for the wearable devices. The traditional FTENGs made of Teflon result in better performance but are not suitable for long-term wear in person. Here, we propose a novel FTENG using a flexible micro-needle-structured polydimethylsiloxane (MN-PDMS) together with the comfortable commercially available 2D-polyester fibers, and electroless nickel-plated cotton cloth of which two are widely used in human daily life. The MN-PDMS is formed by a laser engraved mold for improving its output performance of FTENG compared to the flat-PDMS. The open-circuit voltage (Voc) and the short-circuit current (Isc) of MN-FTENG increased to 73.6 V and 36 μA, respectively, which are 34% and 37% higher than the flat-FTENG. In terms of power, the performance of MN-FTENG reaches 1.296 mW which is 89% higher than that of flat-TENG and it can also light up 90 LEDs. For application, human motion at the joints can be detected and collected with various signals that are used for the human-machine interface (HMI) through the cooperation of components for the Internet of Things (IoT). It can light up the LED bulb through MN-FTENG to potentially develop IoT HMI systems for human motion control of robot in the future.

Smart Contact Lenses for the New Era of IoT: Integrated Biosensors, Circuits, and Human–Machine Interface Systems

AbstractThe rapid development of the Internet of Things (IoT) technology endows some traditional devices with intelligent functions. Compared with traditional contact lenses used for correcting vision or beautifying the eyes, smart contact lenses (SCLs) are developed to monitor some physiological information of the eye. Sculls can be used to continuously monitor eye diseases non‐invasively in real‐time. As a personal electronic device, SCs can aid people in understanding their physical condition better without affecting their personal life. This review mainly discusses the development direction and problems associated with sculls from the perspective of the IoT. SCs consist of three main parts: biosensors, circuits, and human–machine interface (HMI) systems. These three parts correspond to the sensing, connection, and application layer of the IoT. Finally, the problems that may be encountered in the development of SCLs are discussed and potential solutions are proposed.

Crocodile-Skin-Inspired Omnidirectionally Stretchable Pressure Sensor.

Stretchable pressure sensors are important components of multimodal electronic skin needed for potentializing numerous Internet of Things applications. In particular, to use pressure sensors in various wearable/skin-attachable electronics, both high deformability and strain-independent sensitivity must be realized. However, previously reported stretchable pressure sensors cannot meet these standards because they exhibit limited stretchability and nonuniform sensitivity under deformation. Herein, inspired by the unique sensory organ of a crocodile, an omnidirectionally stretchable piezoresistive pressure sensor made of polydimethylsiloxane (PDMS)/silver nanowires (AgNWs) composites with microdomes and wrinkled surfaces is developed. The stretchable pressure sensor exhibits high sensitivity that changes negligibly even under uniaxial and biaxial tensile strains of 100% and 50%, respectively. This behavior is attributed to the microdomes responsible for detecting applied pressures being weakly affected by tensile strains, while the isotropic wrinkles between the microdomes deform to effectively reduce the external stress. In addition, because the device comprises all PDMS-based structures, it exhibits outstanding robustness under repeated mechanical stimuli. The device shows strong potential as a wearable pressure sensor and an artificial crocodile sensing organ, successfully detecting applied pressures in various scenarios. Therefore, the pressure sensor is expected to find applications in electronic skin for prosthetics and human-machine interface systems.

Frequency-invariant beamformer design via ADPM approach

Frequency-invariant (FI) beamformers play an important role in suppressing speech waveform distortions. Due to the perfect FI beampattern (FIB), differential microphone arrays (DMAs) have been widely used in practical applications like voice communication and human-machine interface systems. Superdirective beamforming generated by DMAs have many useful properties but suffer white noise amplification. To address this drawback, we formulate a least squares broadband FI problem, under the white noise output power constraints, to improve robustness of superdirective beamformers. The problem is challenging to solve since FI beamformers are designed on broadband and initialization parameters corresponding to each frequency are different. We devise broadband beampattern synthesis algorithm based on alternating direction penalty method (ADPM), which utilizes the relationship between residuals and penalty terms to reduce the iteration number under the improved framework of alternating direction method of multipliers (ADMM). The proposed ADPM method can decompose the optimization problem into multi-block convex optimization problems and solve them separately. The fast convergence property of our solution is demonstrated via numerical simulations.

(Invited) Soft Wearable Biosensors and Bioelectronics for Human Healthcare

In this talk, Dr. Yeo will discuss the fundamental study in soft materials, flexible mechanics, nanomanufacturing, machine learning, and system packaging to develop nanomembrane-based intelligent soft wearable biosensors and bioelectronics. He will also talk about how fundamental science and knowledge can be applied to create various types of wearable soft sensors, circuits, and integrated bioelectronics. Afterward, he will share application examples of the wearable soft system as a portable health monitoring device, disease diagnostic device, therapeutic system, and human-machine interface system. Details of a device design, manufacturing, optimization, signal processing, and classification will be shared at high levels.

Tunable mechanical, self-healing hydrogels driven by sodium alginate and modified carbon nanotubes for health monitoring

Conductive hydrogels featuring a modulus similar to the skin have flourished in health monitoring and human-machine interface systems. However, developing conductive hydrogels with self-healing and tunable force-electrical performance remains a problem. Herein, a hydrogen bonding cross-linking strategy was utilized by incorporating silk sericin-modified carbon nanotubes (SS@CNTs) into sodium alginate (SA) and polyvinyl alcohol (PVA). Hydrogels synthesized with desirable tensile strength and self-healing ability (67.2 % self-healing efficiency in fracture strength) assembled into strain sensors with a low detection limit of 0.5 % and a gauge factor (GF) of 4.75 (0–17 %). Additionally, as-prepared hydrogels exhibit high sensitivity to tiny pressure changes, allowing recognition of complex handwriting. Notably, resulting hydrogels possess self-powered property, generating up to 215 V and illuminating 100 commercial green LEDs. This work stems from the pressing need for multifunctional hydrogels with prospective applications in human motion sensing and energy harvesting.

A smart PLC-SCADA framework for monitoring petroleum products terminals in industry 4.0 via machine learning

The paper introduces a Programmable Logic Controller (PLC) / Human Machine Interface (HMI) system incorporated along with machine learning (ML) classifiers. Although some other studies have incorporated ML techniques to predict and control petroleum product terminals in terms of concentration, the proposed framework incorporates Add On Instruction (AOI) programming, PLC, and ML methods to automatically monitor petroleum products terminals. The framework adds an AOI in programming to achieve maximum usage of processor capabilities. Moreover, it uses AOI for programming in cooperation with the ladder diagram (LD). This leads to simplifying the LD graphical programming language, reducing the time of scanning, and making facilitate troubleshooting. The AOI is merged with ML to automate tank level detection and maintain good operational conditions and consequently protect these expensive essential assets. The introduced framework consists of three stages. The first stage is the PLC programming phase where the PLC is created using Add-On instructions. Next, HMI graphic displays are drawn and linked to the PLC tags in the following stage. During the third stage, the actual process readings are applied to the system based on ML algorithms to test its functionality. The proposed system results indicates a reduction in the LD number, highest program size, and maximum time of scanning. The results indicate that the AOI can help to trace the program more easily in fault situations. Besides, additional program instructions could reduce processor memory, system construction costs, and upgrade projects.

Perancangan Human Machine Interface (HMI) Pada Modul Praktikum Sistem HMI Berbasis PLC Omron CP1E NA20DRA

This study aims to design practical learning media. The Human Machine Interface (HMI) learning module consists of a monitor to display control menus, PLC devices, and input-output devices. The design of the exemplary HMI display is needed to provide the required information to the user. The research method used in this study uses experimental methods and program design on the trainer module. From the study results, it can conclude that the use of the Omron CP1E NA20DRA PLC-Based HMI system is effortless to use as a learning medium and can train students in improving PLC programming skills

Farming‐Inspired Continuous Fabrication of Grating Flexible Transparent Film with Anisotropic Conductivity

AbstractFlexible transparent conductive films (FTCFs) are critically important for the emerging flexible electronic devices. To this end, FTCFs with conductive gratings have drawn more interests in the past decades. However, there are great challenges to fabricate such FTCFs containing conductive gratings in a facile way. Borrowing the idea from traditional farming, a homemade roll‐to‐roll plowing technique combined with blade coating is proposed to fabricate silver grating FTCFs with anisotropic conductivity in a continuous, cost‐effective, and environmental friendly way. The as‐prepared FTCFs exhibit satisfied optical and electrical properties (i.e., transmittance is 78.2% and sheet resistance is 3.1 Ω □–1). After coated with thin polyvinyl butyral layer through hot‐pressing, it shows a higher transmittance of 84.2% and remarkable mechanical flexibility. Based on the FTCFs with anisotropic conductivity, flexible transparent heater and capacitive keyboard are successfully fabricated. Furthermore, based on such flexible capacitive keyboard, smart wireless controlled human–machine interface system is developed to wirelessly control tablet personal computer working and quadrotor flying. This study provides a promising strategy to fabricate high‐performance FTCFs with anisotropic conductivity via a continuous, cost‐effective, and environmental friendly way.